Moving image encoding device, moving image decoding device, moving image coding method, and moving image decoding method

ABSTRACT

When carrying out an intra-frame prediction process to generate an intra prediction image by using an already-encoded image signal in a frame, an intra prediction part  4  selects a filter from one or more filters which are prepared in advance according to the states of various parameters associated with the encoding of a target block to be filtered, and carries out a filtering process on a prediction image by using the filter. As a result, prediction errors which occur locally can be reduced, and the image quality can be improved.

This application is a Divisional of copending application Ser. No.13/822,887, filed on Mar. 13, 2013, which is a 371 National PhaseApplication of PCT/JP2011/004122 which claims priority under 35 U.S.C.§119(a) to Application No. JP2010-221471, filed in Japan on Sep. 30,2010, all of which are hereby expressly incorporated by reference intothe present application.

FIELD OF THE INVENTION

The present invention relates to a moving image encoding device for anda moving image encoding method of encoding a moving image with a highdegree of efficiency, and a moving image decoding device for and amoving image decoding method of decoding an encoded moving image with ahigh degree of efficiency.

BACKGROUND OF THE INVENTION

For example, in accordance with an international standard video encodingmethod, such as MPEG (Moving Picture Experts Group) or “ITU-T H.26x”, aninputted video frame is divided into rectangular blocks (encoding targetblocks), a prediction process using an already-encoded image signal iscarried out on each encoding target block to generate a predictionimage, and orthogonal transformation and a quantization process iscarried out on a prediction error signal which is the difference betweenthe encoding target block and the prediction image in units of a block,so that information compression is carried out on the inputted videoframe.

For example, in the case of AVC/H.264 (ISO/IEC 14496-101 ITU-T H.264)which is an international standard method, an intra prediction processfrom already-encoded adjacent pixels or a motion-compensated predictionprocess between adjacent frames is carried out (for example, refer tononpatent reference 1). In the case of MPEG-4 AVC/H.264, one predictionmode can be selected from a plurality of prediction modes for each blockin an intra prediction mode of luminance. FIG. 10 is an explanatorydrawing showing intra prediction modes in the case of a 4×4 pixel blocksize for luminance. In FIG. 10, each white circle shows a pixel in acoding block, and each black circle shows a pixel that is used forprediction, and that exists in an already-encoded adjacent block.

In the example shown in FIG. 10, nine modes 0 to 8 are prepared as intraprediction modes, and the mode 2 is the one in which an averageprediction is carried out in such a way that each pixel in the targetcoding block is predicted by using the average of adjacent pixelsexisting in the upper and left blocks. The modes other than the mode 2are intra prediction modes in each of which a directional prediction iscarried out. The mode 0 is the one in which a vertical prediction iscarried out in such a way that adjacent pixels in the upper block arerepeatedly replicated to create plural rows of pixels along a verticaldirection to generate a prediction image. For example, the mode 0 isselected when the target coding block is a vertically striped pattern.The mode 1 is the one in which a horizontal prediction is carried out insuch a way that adjacent pixels in the left block are repeatedlyreplicated to create plural columns of pixels along a horizontaldirection to generate a prediction image. For example, the mode 1 isselected when the target coding block is a horizontally striped pattern.In each of the modes 3 to 8, interpolation pixels running in apredetermined direction (i.e., a direction shown by arrows) aregenerated by using the adjacent pixels in the upper block or the leftblock to generate a prediction image.

In this case, the block size for luminance to which an intra predictionis applied can be selected from 4×4 pixels, 8×8 pixels, and 16×16pixels. In the case of 8×8 pixels, nine intra prediction modes aredefined, like in the case of 4×4 pixels. In contrast with this, in thecase of 16×16 pixels, four intra prediction modes which are called planepredictions are defined in addition to intra prediction modes associatedwith an average prediction, a vertical prediction, and a horizontalprediction. Each intra prediction associated with a plane prediction isa mode in which pixels created by carrying out an interpolation in adiagonal direction on the adjacent pixels in the upper block and theadjacent pixels in the left block are provided as predicted values.

In a directional prediction mode in the case of a block size of 4×4pixels or 8×8 pixels, because predicted values are generated along adirection predetermined according to the mode, e.g., a direction of 45degrees, the prediction efficiency increases and the code amount can bereduced when the direction of a boundary (edge) of an object in a blockmatches the direction shown by the prediction mode. However, a slightdisplacement may occur between the direction of an edge and thedirection shown by the prediction mode, and, even if the direction of anedge in the encoding target block does not match the direction shown bythe prediction mode, a large prediction error may occur locally for thesimple reason that the edge is slightly distorted (swung, bent, or thelike). As a result, the prediction efficiency may drop extremely. Inorder to prevent such a reduction in the prediction efficiency, whenperforming an 8×8-pixel directional prediction, a smoothed predictionimage is generated by setting encoded adjacent pixels which are filteredby a smoothing filter as reference images which are used at the time ofgenerating a prediction image, thereby reducing any slight displacementin the prediction direction and prediction errors which occur when aslight distortion occurs in an edge.

RELATED ART DOCUMENT Nonpatent Reference

-   Nonpatent reference 1: MPEG-4 AVC (ISO/IEC 14496-10)/H.ITU-T 264    standards

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

Because the conventional image encoding device is constructed as above,carrying out a filtering process to generate a smoothed prediction imagecan reduce prediction errors occurring even if a slight displacementoccurs in the prediction direction or a slight distortion occurs in anedge. However, according to the technique disclosed in nonpatentreference 1, no filtering process is carried out on blocks other than8×8-pixel blocks, and only one possible filter used for 8×8-pixel blocksis provided. A problem is that also in a block having a size other than8×8 pixels, a large prediction error actually occurs locally due to aslight mismatch in an edge even when the prediction image has a patternsimilar to that of the image to be encoded, and therefore a largereduction occurs in the prediction efficiency. A further problem is thatwhen a quantization parameter which is used when quantizing a predictionerror signal, the position of each pixel, or the like differs betweenblocks having the same size, a filter suitable for reducing localprediction errors differs between the blocks, but only one possiblefilter is prepared, and therefore prediction errors cannot besufficiently reduced.

The present invention is made in order to solve the above-mentionedproblems, and it is therefore an object of the present invention toprovide a moving image encoding device, a moving image decoding device,a moving image encoding method, and a moving image decoding methodcapable of reducing prediction errors which occur locally, thereby beingable to improve the image quality.

Means for Solving the Problem

In accordance with the present invention, there is provided a movingimage decoding device including an intra prediction unit for, when anencoding mode associated with a coding block is an intra encoding mode,carrying out an intra-frame prediction process on each block which is aunit for prediction process of the coding block, in which the intraprediction unit generates an intermediate predicted value from referencesamples according to an intra prediction parameter, sets a value whichis obtained by filtering the intermediate predicted value as a finalpredicted value only at specific positions in the block, and sets theintermediate predicted value as a final predicted value at any otherpositions in the block.

Advantages of the Invention

Because the moving image decoding device in accordance with the presentinvention is constructed in such a way that the moving image decodingdevice includes the intra prediction unit for, when an encoding modeassociated with a coding block is an intra encoding mode, carrying outan intra-frame prediction process on each block which is a unit forprediction process of the coding block, and the intra prediction unitgenerates an intermediate predicted value from reference samplesaccording to an intra prediction parameter, sets a value which isobtained by filtering the intermediate predicted value as a finalpredicted value only at specific positions in the block, and sets theintermediate predicted value as a final predicted value at any otherpositions in the block, there is provided an advantage of being able toreduce prediction errors which occur locally, thereby enabling even themoving image decoding device to generate the same intra prediction imageas that generated by a moving image encoding device having a high degreeof image quality.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram showing a moving image encoding device inaccordance with Embodiment 1 of the present invention;

FIG. 2 is a block diagram showing a moving image decoding device inaccordance with Embodiment 1 of the present invention;

FIG. 3 is a flow chart showing processing carried out by the movingimage encoding device in accordance with Embodiment 1 of the presentinvention;

FIG. 4 is a flow chart showing processing carried out by the movingimage decoding device in accordance with Embodiment 1 of the presentinvention;

FIG. 5 is an explanatory drawing showing a state in which each codingblock having a maximum size is hierarchically divided into a pluralityof coding blocks;

FIG. 6( a) is an explanatory drawing showing a distribution ofpartitions into which a block to encoded is divided, and FIG. 6( b) isan explanatory drawing showing a state in which an encoding modem(B^(n)) is assigned to each of the partitions after a hierarchicallayer division is performed by using a quadtree graph;

FIG. 7 is an explanatory drawing showing an example of intra predictionparameters (intra prediction mode) which can be selected for eachpartition P_(i) ^(n) in a coding block B^(n);

FIG. 8 is an explanatory drawing showing an example of pixels which areused when generating a predicted value of each pixel in a partitionP_(i) ^(n) in the case of l_(i) ^(n)=m_(i) ^(n)=4;

FIG. 9 is an explanatory drawing showing an example of the arrangementof reference pixels in the case of N=5; and

FIG. 10 is an explanatory drawing showing intra prediction modes in thecase of a 4×4 pixel block size for luminance.

EMBODIMENTS OF THE INVENTION

Hereafter, in order to explain this invention in greater detail, thepreferred embodiments of the present invention will be described withreference to the accompanying drawings.

Embodiment 1

In this Embodiment 1, a moving image encoding device that inputs eachframe image of a video, carries out an intra prediction process fromalready-encoded adjacent pixels or a motion-compensated predictionprocess between adjacent frames to generate a prediction image, carriesout a compression process according to orthogonal transformation andquantization on a prediction error signal which is a difference imagebetween the prediction image and a frame image, and, after that, carriesout variable length encoding to generate a bitstream, and a moving imagedecoding device that decodes the bitstream outputted from the movingimage encoding device will be explained.

The moving image encoding device in accordance with this Embodiment 1 ischaracterized in that the moving image encoding device adapts itself toa local change of a video signal in space and time directions to dividethe video signal into regions of various sizes, and carries outintra-frame and inter-frame adaptive encoding. In general, a videosignal has a characteristic of its complexity varying locally in spaceand time. There can be a case in which a pattern having a uniform signalcharacteristic in a relatively large image area, such as a sky image ora wall image, or a pattern having a complicated texture pattern in asmall image area, such as a person image or a picture including a finetexture, also coexists on a certain video frame from the viewpoint ofspace. Also from the viewpoint of time, a relatively large image area,such as a sky image or a wall image, has a small local change in a timedirection in its pattern, while an image of a moving person or objecthas a larger temporal change because its outline has a movement of arigid body and a movement of a non-rigid body with respect to time.

Although a process of generating a prediction error signal having smallsignal power and small entropy by using a temporal and spatialprediction, thereby reducing the whole code amount, is carried out inthe encoding process, the code amount of parameters used for theprediction can be reduced as long as the parameters can be applieduniformly to as large an image signal region as possible. On the otherhand, because the amount of errors occurring in the prediction increaseswhen the same prediction parameters are applied to an image signalpattern having a large change in time and space, the code amount of theprediction error signal cannot be reduced. Therefore, it is desirable toreduce the size of a region which is subjected to the prediction processwhen performing the prediction process on an image signal pattern havinga large change in time and space, thereby reducing the electric powerand entropy of the prediction error signal even though the data volumeof the parameters which are used for the prediction is increased. Inorder to carry out encoding which is adapted for such the typicalcharacteristics of a video signal, the moving image encoding device inaccordance with this Embodiment 1 hierarchically divides each regionhaving a predetermined maximum block size of the video signal intoblocks, and carries out the prediction process and the encoding processof encoding a prediction error on each of the blocks into which eachregion is divided.

A video signal which is to be processed by the moving image encodingdevice in accordance with this Embodiment 1 can be an arbitrary videosignal in which each video frame consists of a series of digital samples(pixels) in two dimensions, horizontal and vertical, such as a YUVsignal which consists of a luminance signal and two color differencesignals, a color video image signal in arbitrary color space, such as anRGB signal, outputted from a digital image sensor, a monochrome imagesignal, or an infrared image signal. The gradation of each pixel can bean 8-bit, 10-bit, or 12-bit one. In the following explanation, theinputted video signal is a YUV signal unless otherwise specified. It isfurther assumed that the two color difference components U and V aresignals having a 4:2:0 format which are subsampled with respect to theluminance component Y. A data unit to be processed which corresponds toeach frame of the video signal is referred to as a “picture.” In thisEmbodiment 1, a “picture” is explained as a video frame signal on whichprogressive scanning has been carried out. When the video signal is aninterlaced signal, a “picture” can be alternatively a field image signalwhich is a unit which constructs a video frame.

FIG. 1 is a block diagram showing a moving image encoding device inaccordance with Embodiment 1 of the present invention. Referring to FIG.1, an encoding controlling part 1 carries out a process of determining amaximum size of each of coding blocks which is a unit to be processed ata time when an intra prediction process (intra-frame prediction process)or a motion-compensated prediction process (inter-frame predictionprocess) is carried out, and also determining an upper limit on thenumber of hierarchical layers in a hierarchy in which each of the codingblocks having the maximum size is hierarchically divided into blocks.The encoding controlling part 1 also carries out a process of selectingan encoding mode suitable for each of the coding blocks into which eachcoding block having the maximum size is divided hierarchically from oneor more available encoding modes (one or more intra encoding modes andone or more inter encoding modes). The encoding controlling part 1further carries out a process of determining a quantization parameterand a transformation block size which are used when a difference imageis compressed for each coding block, and also determining intraprediction parameters or inter prediction parameters which are used whena prediction process is carried out for each coding block. Thequantization parameter and the transformation block size are included inprediction error encoding parameters, and these prediction errorencoding parameters are outputted to a transformation/quantization part7, an inverse quantization/inverse transformation part 8, a variablelength encoding part 13, and so on. The encoding controlling part 1constructs an encoding controlling unit.

A block dividing part 2 carries out a process of, when receiving a videosignal showing an inputted image, dividing the inputted image shown bythe video signal into coding blocks each having the maximum sizedetermined by the encoding controlling part 1, and also dividing each ofthe coding blocks into blocks hierarchically until the number ofhierarchical layers reaches the upper limit on the number ofhierarchical layers which is determined by the encoding controlling part1. The block dividing part 2 constructs a block dividing unit. Aselection switch 3 carries out a process of, when the encoding modeselected by the encoding controlling part 1 for the coding block, whichis generated through the division by the block dividing part 2, is anintra encoding mode, outputting the coding block to an intra predictionpart 4, and, when the encoding mode selected by the encoding controllingpart 1 for the coding block, which is generated through the division bythe block dividing part 2, is an inter encoding mode, outputting thecoding block to a motion-compensated prediction part 5.

The intra prediction part 4 carries out a process of, when receiving thecoding block, which is generated through the division by the blockdividing part 2, from the selection switch 3, carrying out an intraprediction process on the coding block to generate a prediction image byusing an already-encoded image signal in the frame on the basis of theintra prediction parameters outputted thereto from the encodingcontrolling part 1. After generating the above-mentioned predictionimage, the intra prediction part 4 selects a filter from one or morefilters which are prepared in advance according to the states of thevarious parameters associated with the encoding of the target block tobe filtered, carries out a filtering process on the above-mentionedprediction image by using the filter, and outputs the prediction imageon which the intra prediction part has carried out the filtering processto a subtracting part 6 and an adding part 9. The intra prediction partselects the above-mentioned filter in consideration of at least one ofthe following four parameters:

Parameter (1)

The block size of the above-mentioned prediction image

Parameter (2)

The quantization parameter determined by the encoding controlling part 1

Parameter (3)

The distance between the already-encoded image signal in the frame whichis used when generating the prediction image and a target pixel to befiltered

Parameter (4)

The intra prediction parameters determined by the encoding controllingpart 1

An intra prediction unit is comprised of the selection switch 3 and theintra prediction part 4.

The motion-compensated prediction part 5 carries out a process of, whenan inter encoding mode is selected by the encoding controlling part 1 asan encoding mode suitable for the coding block, which is generatedthrough the division by the block dividing part 2, performing amotion-compensated prediction process on the coding block to generate aprediction image by using one or more frames of reference images storedin a motion-compensated prediction frame memory 12 on the basis of theinter prediction parameters outputted thereto from the encodingcontrolling part 1. A motion-compensated prediction unit is comprised ofthe selection switch 3 and the motion-compensated prediction part 5.

The subtracting part 6 carries out a process of subtracting theprediction image generated by the intra prediction part 4 or themotion-compensated prediction part 5 from the coding block, which isgenerated through the division by the block dividing part 2, to generatea difference image (=the coding block−the prediction image). Thesubtracting part 6 constructs a difference image generating unit. Thetransformation/quantization part 7 carries out a process of performing atransformation process (e.g., a DCT (discrete cosine transform) or anorthogonal transformation process, such as a KL transform, in whichbases are designed for a specific learning sequence in advance) on thedifference signal generated by the subtracting part 6 in units of ablock having a transformation block size included in the predictionerror encoding parameters outputted thereto from the encodingcontrolling part 1, and also quantizing the transform coefficients ofthe difference image by using a quantization parameter included in theprediction error encoding parameters to output the transformcoefficients quantized thereby as compressed data of the differenceimage. The transformation/quantization part 7 constructs an imagecompression unit.

The inverse quantization/inverse transformation part 8 carries out aprocess of inverse-quantizing the compressed data outputted thereto fromthe transformation/quantization part 7 by using the quantizationparameter included in the prediction error encoding parameters outputtedthereto from the encoding controlling part 1, and performing an inversetransformation process (e.g., an inverse DCT (inverse discrete cosinetransform) or an inverse transformation process such as an inverse KLtransform) on the compressed data inverse-quantized thereby in units ofa block having the transformation block size included in the predictionerror encoding parameters to output the compressed data on which theinverse quantization/inverse transformation part has carried out theinverse transformation process as a local decoded prediction errorsignal.

The adding part 9 carries out a process of adding the local decodedprediction error signal outputted thereto from the inversequantization/inverse transformation part 8 and the prediction signalshowing the prediction image generated by the intra prediction part 4 orthe motion-compensated prediction part 5 to generate a local decodedimage signal showing a local decoded image. A memory 10 for intraprediction is a recording medium, such as a RAM, for storing the localdecoded image shown by the local decoded image signal generated by theadding part 9 as an image which the intra prediction part 4 will usewhen performing the intra prediction process the next time.

A loop filter part 11 carries out a process of compensating for anencoding distortion included in the local decoded image signal generatedby the adding part 9, and outputting the local decoded image shown bythe local decoded image signal on which the loop filter part has carriedout the encoding distortion compensation to a motion-compensatedprediction frame memory 12 as a reference image. The motion-compensatedprediction frame memory 12 is a recording medium, such as a RAM, forstoring the local decoded image on which the loop filter part 11 hascarried out the filtering process as a reference image which themotion-compensated prediction part 5 will use when performing themotion-compensated prediction process the next time.

The variable length encoding part 13 carries out a process ofvariable-length-encoding the compressed data outputted thereto from thetransformation/quantization part 7, the encoding mode and the predictionerror encoding parameters which are outputted thereto from the encodingcontrolling part 1, and the intra prediction parameters outputtedthereto from the intra prediction part 4 or the inter predictionparameters outputted thereto from the motion-compensated prediction part5 to generate a bitstream into which encoded data of the compresseddata, encoded data of the encoding mode, encoded data of the predictionerror encoding parameters, and encoded data of the intra predictionparameters or the inter prediction parameters are multiplexed. Thevariable length encoding part 13 constructs a variable length encodingunit.

FIG. 2 is a block diagram showing the moving image decoding device inaccordance with Embodiment 1 of the present invention. Referring to FIG.2, a variable length decoding part 51 carries out a process ofvariable-length-decoding the encoded data multiplexed into the bitstreamto obtain the compressed data, the encoding mode, the prediction errorencoding parameters, and the intra prediction parameters or the interprediction parameters, which are associated with each coding block intowhich each frame of the video is hierarchically divided, and outputtingthe compressed data and the prediction error encoding parameters to aninverse quantization/inverse transformation part 55, and also outputtingthe encoding mode and the intra prediction parameters or the interprediction parameters to a selection switch 52. The variable lengthdecoding part 51 constructs a variable length decoding unit.

The selection switch 52 carries out a process of, when the encoding modeassociated with the coding block, which is outputted from the variablelength decoding part 51, is an intra encoding mode, outputting the intraprediction parameters outputted thereto from the variable lengthdecoding part 51 to an intra prediction part 53, and, when the encodingmode is an inter encoding mode, outputting the inter predictionparameters outputted thereto from the variable length decoding part 51to a motion-compensated prediction part 54.

The intra prediction part 53 carries out a process of performing anintra-frame prediction process on the coding block to generate aprediction image by using an already-decoded image signal in the frameon the basis of the intra prediction parameters outputted thereto fromthe selection switch 52. After generating the above-mentioned predictionimage, the intra prediction part 53 selects a filter from one or morefilters which are prepared in advance according to the states of thevarious parameters associated with the decoding of the target block tobe filtered, carries out a filtering process on the above-mentionedprediction image by using the filter, and outputs the prediction imageon which the intra prediction part has carried out the filtering processto an adding part 56. The intra prediction part selects theabove-mentioned filter in consideration of at least one of the followingfour parameters:

Parameter (1)

The block size of the above-mentioned prediction image

Parameter (2)

The quantization parameter variable-length-decoded by the variablelength decoding part 51

Parameter (3)

The distance between the already-decoded image signal in the frame whichis used when generating the prediction image and a target pixel to befiltered

Parameter (4)

The intra prediction parameters variable-length-decoded by the variablelength decoding part 51

An intra prediction unit is comprised of the selection switch 52 and theintra prediction part 53.

The motion-compensated prediction part 54 carries out a process ofperforming a motion-compensated prediction process on the coding blockto generate a prediction image by using one or more frames of referenceimages stored in a motion-compensated prediction frame memory 59 on thebasis of the inter prediction parameters outputted thereto from theselection switch 52. A motion-compensated prediction unit is comprisedof the selection switch 52 and the motion-compensated prediction part54.

The inverse quantization/inverse transformation part 55 carries out aprocess of inverse-quantizing the compressed data associated with thecoding block, which is outputted thereto from the variable lengthdecoding part 51, by using the quantization parameter included in theprediction error encoding parameters outputted thereto from the variablelength decoding part 51, and performing an inverse transformationprocess (e.g., an inverse DCT (inverse discrete cosine transform) or aninverse transformation process such as an inverse KL transform) on thecompressed data inverse-quantized thereby in units of a block having thetransformation block size included in the prediction error encodingparameters, and outputting the compressed data on which the inversequantization/inverse transformation part has carried out the inversetransformation process as a decoded prediction error signal (signalshowing a pre-compressed difference image). The inversequantization/inverse transformation part 55 constructs a differenceimage generating unit.

The adding part 56 carries out a process of adding the decodedprediction error signal outputted thereto from the inversequantization/inverse transformation part 55 and the prediction signalshowing the prediction image generated by the intra prediction part 53or the motion-compensated prediction part 54 to generate a decoded imagesignal showing a decoded image. The adding part 56 constructs a decodedimage generating unit. A memory 57 for intra prediction is a recordingmedium, such as a RAM, for storing the decoded image shown by thedecoded image signal generated by the adding part 56 as an image whichthe intra prediction part 53 will use when performing the intraprediction process the next time.

A loop filter part 58 carries out a process of compensating for anencoding distortion included in the decoded image signal generated bythe adding part 56, and outputting the decoded image shown by thedecoded image signal on which the loop filter part performs the encodingdistortion compensation to a motion-compensated prediction frame memory59 as a reference image. The motion-compensated prediction frame memory59 is a recording medium, such as a RAM, for storing the decoded imageon which the loop filter part 58 performs the filtering process as areference image which the motion-compensated prediction part 54 will usewhen performing the motion-compensated prediction process the next time.

In the example shown in FIG. 1, the encoding controlling part 1, theblock dividing part 2, the selection switch 3, the intra prediction part4, the motion-compensated prediction part 5, the subtracting part 6, thetransformation/quantization part 7, the inverse quantization/inversetransformation part 8, the adding part 9, the loop filter part 11, andthe variable length encoding part 13, which are the components of themoving image encoding device, can consist of pieces of hardware forexclusive use (e.g., integrated circuits in each of which a CPU ismounted, one chip microcomputers, or the like), respectively. As analternative, the moving image encoding device can consist of a computer,and a program in which the processes carried out by the encodingcontrolling part 1, the block dividing part 2, the selection switch 3,the intra prediction part 4, the motion-compensated prediction part 5,the subtracting part 6, the transformation/quantization part 7, theinverse quantization/inverse transformation part 8, the adding part 9,the loop filter part 11, and the variable length encoding part 13 aredescribed can be stored in a memory of the computer and the CPU of thecomputer can be made to execute the program stored in the memory. FIG. 3is a flow chart showing the processing carried out by the moving imageencoding device in accordance with Embodiment 1 of the presentinvention.

In the example shown in FIG. 2, the variable length decoding part 51,the selection switch 52, the intra prediction part 53, themotion-compensated prediction part 54, the inverse quantization/inversetransformation part 55, the adding part 56, and the loop filter part 58,which are the components of the moving image decoding device, canconsist of pieces of hardware for exclusive use (e.g., integratedcircuits in each of which a CPU is mounted, one chip microcomputers, orthe like), respectively. As an alternative, the moving image decodingdevice can consist of a computer, and a program in which the processescarried out by the variable length decoding part 51, the selectionswitch 52, the intra prediction part 53, the motion-compensatedprediction part 54, the inverse quantization/inverse transformation part55, the adding part 56, and the loop filter part 58 are described can bestored in a memory of the computer and the CPU of the computer can bemade to execute the program stored in the memory. FIG. 4 is a flow chartshowing the processing carried out by the moving image decoding devicein accordance with Embodiment 1 of the present invention.

Next, the operation of the moving image encoding device and theoperation of the moving image decoding device will be explained. First,the processing carried out by the moving image encoding device shown inFIG. 1 will be explained. First, the encoding controlling part 1determines a maximum size of each of coding blocks which is a unit to beprocessed at a time when an intra prediction process (intra-frameprediction process) or a motion-compensated prediction process(inter-frame prediction process) is carried out, and also determines anupper limit on the number of hierarchical layers in a hierarchy in whicheach of the coding blocks having the maximum size is hierarchicallydivided into blocks (step ST1 of FIG. 3).

As a method of determining the maximum size of each of coding blocks,for example, there is considered a method of determining a maximum sizefor all the pictures according to the resolution of the inputted image.Further, there can be considered a method of quantifying a variation inthe complexity of a local movement of the inputted image as a parameterand then determining a small size for a picture having a large andvigorous movement while determining a large size for a picture having asmall movement. As a method of determining the upper limit on the numberof hierarchical layers, for example, there can be considered a method ofincreasing the depth of the hierarchy, i.e., the number of hierarchicallayers to make it possible to detect a finer movement as the inputtedimage has a larger and more vigorous movement, or decreasing the depthof the hierarchy, i.e., the number of hierarchical layers as theinputted image has a smaller movement.

The encoding controlling part 1 also selects an encoding mode suitablefor each of the coding blocks into which each coding block having themaximum size is divided hierarchically from one or more availableencoding modes (M intra encoding modes and N inter encoding modes) (stepST2). Although a detailed explanation of the selection method ofselecting an encoding mode for use in the encoding controlling part 1will be omitted because the selection method is a known technique, thereis a method of carrying out an encoding process on the coding block byusing an arbitrary available encoding mode to examine the encodingefficiency and select an encoding mode having the highest level ofencoding efficiency from among a plurality of available encoding modes,for example.

The encoding controlling part 1 further determines a quantizationparameter and a transformation block size which are used when adifference image is compressed for each coding block, and alsodetermines intra prediction parameters or inter prediction parameterswhich are used when a prediction process is carried out. The encodingcontrolling part 1 outputs prediction error encoding parametersincluding the quantization parameter and the transformation block sizeto the transformation/quantization part 7, the inversequantization/inverse transformation part 8, and the variable lengthencoding part 13. The encoding controlling part also outputs theprediction error encoding parameters to the intra prediction part 4 asneeded.

When receiving the video signal showing the inputted image, the blockdividing part 2 divides the inputted image shown by the video signalinto coding blocks each having the maximum size determined by theencoding controlling part 1, and also divides each of the coding blocksinto blocks hierarchically until the number of hierarchical layersreaches the upper limit on the number of hierarchical layers which isdetermined by the encoding controlling part 1. FIG. 5 is an explanatorydrawing showing a state in which each coding block having the maximumsize is hierarchically divided into a plurality of coding blocks. In theexample of FIG. 5, each coding block having the maximum size is a codingblock B₀ in the 0th hierarchical layer, and its luminance component hasa size of (L⁰, M⁰). Further, in the example of FIG. 5, by carrying outthe hierarchical division with this coding block B₀ having the maximumsize being set as a starting point until the depth of the hierarchyreaches a predetermined depth which is set separately according to aquadtree structure, coding blocks B^(n) can be obtained.

At the depth of n, each coding block B^(n) is an image area having asize of (L^(n), M^(n)). Although L^(n) can be the same as or differ fromM^(n), the case of L^(n)=M^(n) is shown in the example of FIG. 5.Hereafter, the size of each coding block B^(n) is defined as the size of(L^(n), M^(n)) in the luminance component of the coding block B^(n).

Because the block dividing part 2 carries out a quadtree division,(L^(n+1), M^(n+1))=(L^(n)/M^(n)/2) is always established. In the case ofa color video image signal (4:4:4 format) in which all the colorcomponents have the same sample number, such as an RGB signal, all thecolor components have a size of (L^(n), M^(n)), while in the case ofhandling a 4:2:0 format, a corresponding color difference component hasan encoding block size of (L^(n)/2, M^(n)/2). Hereafter, an encodingmode selectable for each coding block B^(n) in the nth hierarchicallayer is expressed as m(B^(n)).

In the case of a color video signal which consists of a plurality ofcolor components, the encoding mode m(B^(n)) can be formed in such a waythat an individual mode is used for each color component. Hereafter, anexplanation will be made by assuming that the encoding mode m(B^(n))indicates the one for the luminance component of each coding blockhaving a 4:2:0 format in a YUV signal unless otherwise specified. Theencoding mode m(B^(n)) can be one of one or more intra encoding modes(generically referred to as “INTRA”) or one or more inter encoding modes(generically referred to as “INTER”), and the encoding controlling part1 selects, as the encoding mode m(B^(n)), an encoding mode with thehighest degree of encoding efficiency for each coding block B^(n) fromamong all the encoding modes available in the picture currently beingprocessed or a subset of these encoding modes, as mentioned above.

Each coding block B^(n) is further divided into one or more predictionunits (partitions) by the block dividing part, as shown in FIG. 5.Hereafter, each partition belonging to each coding block B^(n) isexpressed as P_(i) ^(n) (i shows a partition number in the nthhierarchical layer). How the division of each coding block B^(n) intopartitions P_(i) ^(n) belonging to the coding block B^(n) is carried outis included as information in the encoding mode m(B^(n)). While theprediction process is carried out on each of all the partitions P_(i)^(n) according to the encoding mode m(B^(n)), an individual predictionparameter can be selected for each partition P_(i) ^(n).

The encoding controlling part 1 produces such a block division state asshown in, for example, FIGS. 6( a) and 6(b) for a coding block havingthe maximum size, and then determines coding blocks B^(n). Hatchedportions shown in FIG. 6( a) show a distribution of partitions intowhich the coding block having the maximum size is divided, and FIG. 6(b) shows a situation in which encoding modes m(B^(n)) are respectivelyassigned to the partitions generated through the hierarchical layerdivision by using a quadtree graph. Each node enclosed by □ shown inFIG. 6( b) is a node (coding block B^(n)) to which an encoding modem(B^(n)) is assigned.

When the encoding controlling part 1 selects an optimal encoding modem(B^(n)) for each partition P_(i) ^(n) of each coding block B^(n), andthe encoding mode m(B^(n)) is an intra encoding mode (step ST3), theselection switch 3 outputs the partition P_(i) ^(n) of the coding blockB^(n), which is generated through the division by the block dividingpart 2, to the intra prediction part 4. In contrast, when the encodingmode m (B^(n)) is an inter encoding mode (step ST3), the selectionswitch outputs the partition P_(i) ^(n) of the coding block B^(n), whichis generated through the division by the block dividing part 2, to themotion-compensated prediction part 5.

When receiving the partition P_(i) ^(n) of the coding block B^(n) fromthe selection switch 3, the intra prediction part 4 carries out an intraprediction process on the partition P_(i) ^(n) of the coding block B^(n)to generate an intra prediction image P_(i) ^(n) by using analready-encoded image signal in the frame on the basis of the intraprediction parameters outputted thereto from the encoding controllingpart 1 (step ST4). After generating the above-mentioned intra predictionimage P_(i) ^(n), the intra prediction part 4 selects a filter from oneor more filters which are prepared in advance according to the states ofthe various parameters associated with the encoding of the target blockto be filtered, and carries out a filtering process on the intraprediction image P_(i) ^(n) by using the filter. After carrying out thefiltering process on the intra prediction image P_(i) ^(n), the intraprediction part 4 outputs the intra prediction image P_(i) ^(n) on whichthe intra prediction part has carried out the filtering process to thesubtracting part 6 and the adding part 9. In order to enable the movingimage decoding device shown in FIG. 2 to also be able to generate thesame intra prediction image P_(i) ^(n), the intra prediction partoutputs the intra prediction parameters to the variable length encodingpart 13. The outline of the process carried out by the intra predictionpart 4 is as mentioned above, and the details of this process will bementioned below.

When receiving the partition P_(i) ^(n) of the coding block B^(n) fromthe selection switch 3, the motion-compensated prediction part 5 carriesout a motion-compensated prediction process on the partition P_(i) ^(n)of the coding block B^(n) to generate an inter prediction image P_(i)^(n) by using one or more frames of reference images stored in themotion-compensated prediction frame memory 12 on the basis of the interprediction parameters outputted thereto from the encoding controllingpart 1 (step ST5). Because a technology of carrying out amotion-compensated prediction process to generate a prediction image isknown, the detailed explanation of this technology will be omittedhereafter.

After the intra prediction part 4 or the motion-compensated predictionpart 5 generates the prediction image (an intra prediction image P_(i)^(n) or an inter prediction image P_(i) ^(n)), the subtracting part 6subtracts the prediction image (the intra prediction image P_(i) ^(n) orthe inter prediction image P_(i) ^(n)) generated by the intra predictionpart 4 or the motion-compensated prediction part 5 from the partitionP_(i) ^(n) of the coding block B^(n), which is generated through thedivision by the block dividing part 2, to generate a difference image,and outputs a prediction error signal e_(i) ^(n) showing the differenceimage to the transformation/quantization part 7 (step ST6).

When receiving the prediction error signal e_(i) ^(n) showing thedifference image from the subtracting part 6, thetransformation/quantization part 7 carries out a transformation process(e.g., a DCT (discrete cosine transform) or an orthogonal transformationprocess, such as a KL transform, in which bases are designed for aspecific learning sequence in advance) on the difference image in unitsof a block having the transformation block size included in theprediction error encoding parameters outputted thereto from the encodingcontrolling part 1, and quantizes the transform coefficients of thedifference image by using the quantization parameter included in theprediction error encoding parameters and outputs the transformcoefficients quantized thereby to the inverse quantization/inversetransformation part 8 and the variable length encoding part 13 ascompressed data of the difference image (step ST7).

When receiving the compressed data of the difference image from thetransformation/quantization part 7, the inverse quantization/inversetransformation part 8 inverse-quantizes the compressed data of thedifference image by using the quantization parameter included in theprediction error encoding parameters outputted thereto from the encodingcontrolling part 1, performs an inverse transformation process (e.g., aninverse DCT (inverse discrete cosine transform) or an inversetransformation process such as an inverse KL transform) on thecompressed data inverse-quantized thereby in units of a block having thetransformation block size included in the prediction error encodingparameters, and outputs the compressed data on which the inversequantization/inverse transformation part has carried out the inversetransformation process to the adding part 9 as a local decodedprediction error signal e_(i) ^(n) hat (“A” attached to an alphabeticalletter is expressed by hat for reasons of the restrictions on electronicapplications) (step ST8).

When receiving the local decoded prediction error signal e_(i) ^(n) hatfrom the inverse quantization/inverse transformation part 8, the addingpart 9 adds the local decoded prediction error signal e_(i) ^(n) hat andthe prediction signal showing the prediction image (the intra predictionimage P_(i) ^(n) or the inter prediction image P_(i) ^(n)) generated bythe intra prediction part 4 or the motion-compensated prediction part 5to generate a local decoded image which is a local decoded partitionimage P_(i) ^(n) hat or a local decoded coding block image which is agroup of local decoded partition images (step ST9). After generating thelocal decoded image, the adding part 9 stores a local decoded imagesignal showing the local decoded image in the memory 10 for intraprediction and also outputs the local decoded image signal to the loopfilter part 11.

The moving image encoding device repeatedly carries out the processes ofsteps ST3 to ST9 until the moving image encoding device completes theprocessing on all the coding blocks B^(n) into which the inputted imageis divided hierarchically, and, when completing the processing on allthe coding blocks B^(n), shifts to a process of step ST12 (steps ST10and ST11).

The variable length encoding part 13 entropy-encodes the compressed dataoutputted thereto from the transformation/quantization part 7, theencoding mode (including the information showing the state of thedivision into the coding blocks) and the prediction error encodingparameters, which are outputted thereto from the encoding controllingpart 1, and the intra prediction parameters outputted thereto from theintra prediction part 4 or the inter prediction parameters outputtedthereto from the motion-compensated prediction part 5. The variablelength encoding part 13 multiplexes encoded data which are the encodedresults of the entropy encoding of the compressed data, the encodingmode, the prediction error encoding parameters, and the intra predictionparameters or the inter prediction parameters to generate a bitstream(step ST12).

When receiving the local decoded image signal from the adding part 9,the loop filter part 11 compensates for an encoding distortion includedin the local decoded image signal, and stores the local decoded imageshown by the local decoded image signal on which the loop filter partperforms the encoding distortion compensation in the motion-compensatedprediction frame memory 12 as a reference image (step ST13). The loopfilter part 11 can carry out the filtering process for each coding blockhaving the maximum size of the local decoded image signal outputtedthereto from the adding part 9 or for each coding block of the localdecoded image signal, or for each unit which is a combination of aplurality of coding blocks each having the maximum size. As analternative, after one picture of local decoded image signals isoutputted, the loop filter part can carry out the filtering process onthe picture of local decoded image signals at a time.

Next, the process carried out by the intra prediction unit 4 will beexplained in detail. FIG. 7 is an explanatory drawing showing an exampleof the intra prediction parameters (intra prediction mode) which can beselected for each partition P_(i) ^(n) in the coding block B^(n). In theexample shown in FIG. 7, intra prediction modes and prediction directionvectors represented by each of the intra prediction modes are shown, andit is designed that a relative angle between prediction directionvectors becomes small with increase in the number of selectable intraprediction modes.

The intra prediction part 4 carries out an intra prediction process onthe partition P_(i) ^(n) on the basis of the intra prediction parametersfor the partition P_(i) ^(n) and a selection parameter for a filterwhich the intra prediction part uses for the generation of an intraprediction image P_(i) ^(n). Hereafter, an intra process of generatingan intra prediction signal of the luminance signal on the basis of theintra prediction parameters (intra prediction mode) for the luminancesignal of the partition P_(i) ^(n) will be explained.

Hereafter, the partition P_(i) ^(n) is assumed to have a size of l_(i)^(n)×m_(i) ^(n) pixels. FIG. 8 is an explanatory drawing showing anexample of pixels which are used when generating a predicted value ofeach pixel in the partition P_(i) ^(n) in the case of l_(i) ^(n)=m_(i)^(n)=4. Although the (2×l_(i) ^(n)+1) pixels in the already-encodedupper partition which is adjacent to the partition P_(i) ^(n) and the(2×m_(i) ^(n)) pixels in the already-encoded left partition which isadjacent to the partition P_(i) ^(n) are set as the pixels used forprediction in the example of FIG. 8, a larger or smaller number ofpixels than the pixels shown in FIG. 8 can be used for prediction.Further, although one row or column of pixels adjacent to the partitionare used for prediction in the example shown in FIG. 8, two or more rowsor columns of pixels adjacent to the partition can be alternatively usedfor prediction.

When the index value indicating the intra prediction mode for thepartition P_(i) ^(n) is 2 (average prediction), the intra predictionpart generates an intermediate prediction image by using the average ofthe adjacent pixels in the upper partition and the adjacent pixels inthe left partition as the predicted value of each pixel in the partitionP_(i) ^(n). When the index value indicating the intra prediction mode isother than 2 (average prediction), the intra prediction part generatesthe predicted value of each pixel in the partition P_(i) ^(n) on thebasis of a prediction direction vector v_(p)=(dx, dy) shown by the indexvalue. In this case, the relative coordinate of the pixel (the pixel atthe upper left corner of the partition is set as the point of origin)for which the predicted value is to be generated (target pixel forprediction) in the partition P_(i) ^(n) is expressed as (x, y). Eachreference pixel which is used for prediction is located at a point ofintersection of A shown below and an adjacent pixel.

$L = {\begin{pmatrix}x \\y\end{pmatrix} + {k\; \upsilon_{p}}}$

where k is a positive scalar value.

When a reference pixel is located at an integer pixel position, theinteger pixel is set as the predicted value of the target pixel forprediction. In contrast, when a reference pixel is not located at aninteger pixel position, an interpolation pixel which is generated froman integer pixel adjacent to the reference pixel is set as the predictedvalue of the target pixel for prediction. In the example shown in FIG.8, because a reference pixel is not located at an integer pixelposition, the predicted value is interpolated from the values of twopixels adjacent to the reference pixel. However, the interpolation ofthe predicted value is not limited to the one from the values of twoadjacent pixels, and an interpolation pixel can be generated from two ormore adjacent pixels and the value of this interpolation pixel can beset as the predicted value.

Next, the intra prediction part obtains a final prediction image bycarrying out a filtering process on the intermediate prediction image(predicted value) generated according to the above-mentioned procedure.Hereafter, the filtering process will be explained concretely.

The intra prediction part selects a filter to be used from one or morefilters which are prepared in advance by using a method which will bementioned below, and carries out a filtering process on each pixel ofthe intermediate prediction image according to the following equation(1).

{circumflex over (s)}(p ₀)=a ₀ s(p ₀)+a ₁ s(p ₁)+ . . . +a _(N-1) s(p_(N-1))+a _(N)  (1)

In the equation (1), a_(n) (n=0, 1, N) is filter coefficients whichconsist of coefficients (a₀, a₁, . . . , a_(N-1)) associated with thereference pixels, and an offset coefficient a_(N). p_(n) (n=0, 1, . . ., N−1) shows the reference pixels of the filter including the targetpixel p₀ to be filtered. s(p_(n)) shows the luminance value of eachreference pixel, and s hat (p₀) shows the luminance value of the targetpixel p₀ to be filtered on which the filtering process has been carriedout. The filter coefficients can be formed so as not to include theoffset coefficient a_(N). Further, N is an arbitrary number of referencepixels. FIG. 9 is an explanatory drawing showing an example of thearrangement of the reference pixels in the case of N=5.

When carrying out the above-mentioned filtering process, a nonlinearedge or the like occurs in the inputted image more easily and hence adisplacement from the prediction direction of the intermediateprediction image occurs more easily with increase in the size (l_(i)^(n)×m_(i) ^(n)) of the partition P_(i) ^(n). Therefore, it ispreferable to smooth the intermediate prediction image. In addition, thelarger quantized value a prediction error has, the larger quantizationdistortion occurs in the decoded image and hence the lower degree ofprediction accuracy the inter mediate prediction image generated fromalready-encoded pixels which are adjacent to the partition P_(i) ^(n)has. Therefore, it is preferable to prepare a smoothed prediction imagewhich roughly expresses the partition P_(i) ^(n). Further, even a pixelin the same partition P_(i) ^(n) has a displacement, such as an edge,occurring between the intermediate prediction image and the inputtedimage more easily with distance from the already-encoded pixels adjacentto the partition P_(i) ^(n) which are used for the generation of theintermediate prediction image. Therefore, it is preferable to smooth theprediction image to suppress the rapid increase in the prediction errorwhich is caused when a displacement occurs. In addition, it is necessaryto not only change the intensity of the filter, but also arrange thereference pixels of the filter appropriately according to the predictiondirection of the intermediate prediction image, thereby preventing apattern, such as an edge of the intermediate prediction image, frombeing distorted unnaturally.

Therefore, the filter selecting process is configured in such a way asto select a filter in consideration of the four following parameters (1)to (4).

(1) The size of the partition P_(i) ^(n) (l_(i) ^(n)×m_(i) ^(n))(2) The quantization parameter included in the prediction error encodingparameters(3) The distance between the group of already-encoded pixels (“pixelswhich are used for prediction” shown in FIG. 8) which are used at thetime of generating the intermediate prediction image, and the targetpixel to be filtered(4) The index value indicating the intra prediction mode at the time ofgenerating the intermediate prediction image

More specifically, the filter selecting process is configured in such away that a filter having a higher degree of smoothing intensity is usedwith increase in the size (l_(i) ^(n)×m_(i) ^(n)) of the partition P_(i)^(n), with increase in the quantized value determined by thequantization parameter, and with distance between the target pixel to befiltered and the group of already-encoded pixels which are located onthe left side and on the upper side of the partition P_(i) ^(n) andwhich are used at the time of generating the intermediate predictionimage, and the filer has a degree of filter intensity which isdetermined in consideration of the prediction direction in the intraprediction mode and the reference pixels are arranged in considerationof the prediction direction in the intra prediction mode. Morespecifically, an adaptive selection of a filter according to theabove-mentioned parameters is implemented by bringing an appropriatefilter selected from among the group of filters which are prepared inadvance into correspondence with each of combinations of theabove-mentioned parameters. However, any number of selectable degrees offilter intensity can be provided as long as the number is two or more,and a filtering process equivalent to no filtering can be defined as anexpression of a filter having the lowest degree of smoothing intensity.Therefore, the filtering process can be configured in such a way thatthe filtering process is carried out only on specific pixels in theintermediate prediction image, but a filtering process having the lowestdegree of smoothing intensity, i.e., no filtering is carried out on anyother pixels. Although the above explanation is made on the assumptionthat a necessary number of filters are prepared in advance, a filter canbe alternatively defined as a function of the above-mentioned filterselection parameters in such a way that the filter is determinedaccording to the values of the above-mentioned filter selectionparameters.

Although the example of selecting a filter in consideration of the fourparameters (1) to (4) is shown in the above explanation, a filter can bealternatively selected in consideration of at least one of the fourparameters (1) to (4). In a case of taking into consideration (3) and(4) of the above-mentioned four parameters as an example, there can beprovided a structure of selecting a filter having a higher degree ofintensity with distance from a pixel used for prediction of each targetpixel to be filtered according to the prediction direction in the intraprediction mode (distance from a “reference pixel” which is adjacent tothe upper end of the block in the example shown in FIG. 8). Further,because the four parameters (1) to (4) are known in the moving imagedecoding device, carrying out the above-mentioned filtering processcauses no additional information to be encoded.

The intra prediction part generates a prediction pixel for each of allthe pixels of the luminance signal in the partition P_(i) ^(n) accordingto the same procedure to generate an intra prediction image P_(i) ^(n),and outputs the intra prediction image P_(i) ^(n) generated thereby. Theintra prediction part outputs the intra prediction parameters used forthe generation of the intra prediction image Pi to the variable lengthencoding part 13 in order to multiplex them into a bitstream. The intraprediction part also carries out an intra prediction process based onthe intra prediction parameters (intra prediction mode) on each of thecolor difference signals of the partition P_(i) ^(n) according to thesame procedure as that according to which the intra prediction partcarries out the intra prediction process on the luminance signal, andoutputs the intra prediction parameters used for the generation of theintra prediction image to the variable length encoding part 13. Theintra prediction part can be constructed in such a way as to carry outthe above-explained filtering process for the intra prediction of eachof the color difference signals in the same way that the intraprediction part does for the luminance signal, or not to carry out theabove-explained filtering process for the intra prediction of each ofthe color difference signals.

Next, the processing carried out by the moving image decoding deviceshown in FIG. 2 will be explained. When receiving the bitstreamoutputted thereto from the image encoding device of FIG. 1, the variablelength decoding part 51 carries out a variable length decoding processon the bitstream to decode information having a frame size in units of asequence which consists of one or more frames of pictures or in units ofa picture (step ST21 of FIG. 4). The variable length decoding part 51determines a maximum size of each of coding blocks which is a unit to beprocessed at a time when an intra prediction process (intra-frameprediction process) or a motion-compensated prediction process(inter-frame prediction process) is carried out according to the sameprocedure as that which the encoding controlling part 1 shown in FIG. 1uses, and also determines an upper limit on the number of hierarchicallayers in a hierarchy in which each of the coding blocks having themaximum size is hierarchically divided into blocks (step ST22). Forexample, when the maximum size of each of coding blocks is determinedaccording to the resolution of the inputted image in the image encodingdevice, the variable length decoding part determines the maximum size ofeach of the coding blocks on the basis of the frame size informationwhich the variable length decoding part has decoded previously. Wheninformation showing both the maximum size of each of the coding blocksand the upper limit on the number of hierarchical layers is multiplexedinto the bitstream, the variable length decoding part refers to theinformation which is obtained by decoding the bitstream.

Because the information showing the state of the division of each of thecoding blocks B₀ having the maximum size is included in the encodingmode m(B₀) of the coding block B₀ having the maximum size which ismultiplexed into the bitstream, the variable length decoding part 51specifies each of the coding blocks B^(n) into which the image isdivided hierarchically by decoding the bitstream to obtain the encodingmode m(B₀) of the coding block B₀ having the maximum size which ismultiplexed into the bitstream (step ST23). After specifying each of thecoding blocks B^(n), the variable length decoding part 51 decodes thebitstream to obtain the encoding mode m(B^(n)) of the coding block B^(n)to specify each partition P_(i) ^(n) belonging to the coding block B^(n)on the basis of the info′ illation about the partition P_(i) ^(n)belonging to the encoding mode m(B^(n)). After specifying each partitionP_(i) ^(n) belonging to the coding block B^(n), the variable lengthdecoding part 51 decodes the encoded data to obtain the compressed data,the encoding mode, the prediction error encoding parameters, and theintra prediction parameters/inter prediction parameters for eachpartition P_(i) ^(n) (step ST24).

More specifically, when the encoding mode m(B^(n)) assigned to thecoding block 13 n is an intra encoding mode, the variable lengthdecoding part decodes the encoded data to obtain the intra predictionparameters for each partition P_(i) ^(n) belonging to the coding block.In contrast, when the encoding mode m(B^(n)) assigned to the codingblock B^(n) is an inter encoding mode, the variable length decoding partdecodes the encoded data to obtain the inter prediction parameters foreach partition P_(i) ^(n) belonging to the coding block. The variablelength decoding part further divides each partition which is aprediction unit into one or more partitions which is a transformationprocess unit on the basis of the transformation block size informationincluded in the prediction error encoding parameters, and decodes theencoded data of each of the one or more partitions which is atransformation process unit to obtain the compressed data (transformcoefficients on which transformation and quantization are carried out)of the partition.

When the encoding mode m(B^(n)) of the partition P_(i) ^(n) belonging tothe coding block B^(n), which is specified by the variable lengthdecoding part 51, is an intra encoding mode (step ST25), the selectionswitch 52 outputs the intra prediction parameters outputted thereto fromthe variable length decoding part 51 to the intra prediction part 53. Incontrast, when the encoding mode m(B^(n)) of the partition P_(i) ^(n) isan inter encoding mode (step ST25), the selection switch outputs theinter prediction parameters outputted thereto from the variable lengthdecoding part 51 to the motion-compensated prediction part 54.

When receiving the intra prediction parameters from the selection switch52, the intra prediction part 53 carries out an intra-frame predictionprocess on the partition P_(i) ^(n) of the coding block B^(n) togenerate an intra prediction image P_(i) ^(n) by using analready-decoded image signal in the frame on the basis of the intraprediction parameters (step ST26), like the intra prediction part 4shown in FIG. 1. When generating an intra prediction image P_(i) ^(n),the intra prediction part 53 selects a filter from one or more filters,which are prepared in advance by using the same method as that the intraprediction part 4 shown in FIG. 1 uses, according to the states of thevarious parameters associated with the decoding of the target block tobe filtered, and carries out a filtering process on the intra predictionimage P_(i) ^(n) by using the filter and sets the intra prediction imageP_(i) ^(n) on which the intra prediction part has carried out thefiltering process as a final intra prediction image. Although the aboveexplanation is made on the assumption that a necessary number of filtersare prepared in advance, in the case in which a filter is defined as afunction of the above-mentioned parameters in such a way that the filteris determined according to the states of the parameters used for thefilter selection in the intra prediction part 4 shown in FIG. 1, afilter can be defined as a function of the above-mentioned parametersalso in the intra prediction part 53 in such a way that the filter isdetermined according to the states of the various parameters associatedwith the decoding of the target block to be filtered.

When receiving the inter prediction parameters from the selection switch52, the motion-compensated prediction part 54 carries out anmotion-compensated prediction process on the partition P_(i) ^(n) of thecoding block B^(n) to generate an inter prediction image P_(i) ^(n) byusing one or more frames of reference images stored in themotion-compensated prediction frame memory 59 on the basis of the interprediction parameters (step ST27).

The inverse quantization/inverse transformation part 55inverse-quantizes the compressed data associated with the coding block,which are outputted thereto from the variable length decoding part 51,by using the quantization parameter included in the prediction errorencoding parameters outputted thereto from the variable length decodingpart 51, and carries out an inverse transformation process (e.g., aninverse DCT (inverse discrete cosine transform) or an inversetransformation process such as an inverse KL transform) on thecompressed data inverse-quantized thereby in units of a block having thetransformation block size included in the prediction error encodingparameters, and outputs the compressed data on which the inversequantization/inverse transformation part has carried out the inversetransformation process to the adding part 56 as a decoded predictionerror signal (signal showing a pre-compressed difference image) (stepST28).

When receiving the decoded prediction error signal from the inversequantization/inverse transformation part 55, the adding part 56generates a decoded image by adding the decoded prediction error signaland the prediction signal showing the prediction image generated by theintra prediction part 53 or the motion-compensated prediction part 54and stores a decoded image signal showing the decoded image in thememory 57 for intra prediction, and also outputs the decoded imagesignal to the loop filter part 58 (step ST29).

The moving image decoding device repeatedly carries out the processes ofsteps ST23 to ST29 until the moving image decoding device completes theprocessing on all the coding blocks B″ into which the image is dividedhierarchically (step ST30). When receiving the decoded image signal fromthe adding part 56, the loop filter part 58 compensates for an encodingdistortion included in the decoded image signal, and stores the decodedimage shown by the decoded image signal on which the loop filter partperforms the encoding distortion compensation in the motion-compensatedprediction frame memory 59 as a reference image (step ST31). The loopfilter part 58 can carry out the filtering process for each coding blockhaving the maximum size of the local decoded image signal outputtedthereto from the adding part 56 or each coding block. As an alternative,after the local decoded image signal corresponding to all themacroblocks of one screen is outputted, the loop filter part can carryout the filtering process on all the macroblocks of the one screen at atime.

As can be seen from the above description, because the intra predictionpart 4 of the moving image encoding device in accordance with thisEmbodiment 1 is constructed in such a way as to, when carrying out anintra-frame prediction process to generate an intra prediction image byusing an already-encoded image signal in a frame, select a filter fromone or more filters which are prepared in advance according to thestates of various parameters associated with the encoding of a targetblock to be filtered, and carry out a filtering process on a predictionimage by using the filter, there is provided an advantage of being ableto reduce prediction errors which occur locally, thereby being able toimprove the image quality.

Further, because the intra prediction part 4 in accordance with thisEmbodiment 1 is constructed in such a way as to select a filter inconsideration of at least one of the following parameters: (1) the sizeof the partition P_(i) ^(n) (l_(i) ^(n)×m_(i) ^(n)); (2) thequantization parameter included in the prediction error encodingparameters; (3) the distance between the group of already-encoded pixelswhich are used at the time of generating the intermediate predictionimage, and the target pixel to be filtered; and (4) the index valueindicating the intra prediction mode at the time of generating theintermediate prediction image, there is provided an advantage ofpreventing a local prediction error from occurring when a slightdisplacement occurs between the direction of an edge in the image to beencoded and the prediction direction or a slight distortion exists in anedge in the intermediate prediction image having a high correlation withthe image to be encoded, thereby being able to improve the predictionefficiency.

Because the intra prediction part 53 of the moving image decoding devicein accordance with this Embodiment 1 is constructed in such a way as to,when carrying out an intra-frame prediction process to generate an intraprediction image by using an already-decoded image signal in a frame,select a filter from one or more filters which are prepared in advanceaccording to the states of various parameters associated with thedecoding of a target block to be filtered, and carry out a filteringprocess on a prediction image by using the filter, there is provided anadvantage of reducing prediction errors which occur locally while makingit possible for the moving image decoding device to also generate thesame intra prediction image as that generated by the moving imageencoding device.

Further, because the intra prediction part 53 in accordance with thisEmbodiment 1 is constructed in such a way as to select a filter inconsideration of at least one of the following parameters: (1) the sizeof the partition P_(i) ^(n) (l_(i) ^(n)×m_(i) ^(n)); (2) thequantization parameter included in the prediction error encodingparameters; (3) the distance between the group of already-encoded pixelswhich are used at the time of generating the intermediate predictionimage, and the target pixel to be filtered; and (4) the index valueindicating the intra prediction mode at the time of generating theintermediate prediction image, there are provided an advantage ofpreventing a local prediction error from occurring when a slightdisplacement occurs between the direction of an edge in the image to beencoded and the prediction direction or a slight distortion exists in anedge in the intermediate prediction image having a high correlation withthe image to be encoded, and another advantage of making it possible forthe moving image decoding device to also generate the same intraprediction image as that generated by the moving image encoding device.

Embodiment 2

Although the example in which the intra prediction part 4 selects afilter according to the states of various parameters associated with theencoding of a target block to be filtered from one or more filters whichare prepared in advance, and carries out a filtering process on aprediction image by using the filter when carrying out an intra-frameprediction process to generate an intra prediction image by using analready-encoded image signal in a frame is shown in above-mentionedEmbodiment 1, as an alternative, a Wiener filter which minimizes the sumof squared errors between a coding block and a prediction image can bedesigned, and, when the use of this Wiener filter increases the degreeof reduction in prediction errors as compared with the use of the filterwhich has been selected from the one or more filters which are preparedin advance, the filtering process can be carried out on the predictionimage by using the above-mentioned Wiener filter, instead of the filterwhich has been selected. Hereafter, processes will be explainedconcretely.

Each of the intra prediction parts 4 and 53 in accordance withabove-mentioned Embodiment 1 is constructed in such a way as to select afilter from one or more filters which are prepared in advance accordingto the states of various parameters associated with the encoding of atarget block to be filtered. While each of the intra prediction partscan select an appropriate filter from the one or more selectioncandidates in consideration of the four parameters (1) to (4), each ofthe intra prediction parts cannot carry out “optimal filtering” when anoptimal filter other than the one or more selection candidates exists.This Embodiment 2 is characterized in that while a moving image encodingdevice designs an optimal filter on a per picture basis and carries outa filtering process, and also encodes the filter coefficients of thefilter, and so on, a moving image decoding device decodes the filtercoefficients and so on, and carries out a filtering process by using thefilter.

An intra prediction part 4 of the moving image encoding device carriesout an intra-frame prediction process on each partition P_(i) ^(n) ofeach coding block B^(n) to generate an intra prediction image P_(i)^(n), like that according to above-mentioned Embodiment 1. The intraprediction part 4 also selects a filter from one or more filters whichare prepared in advance according to the states of various parametersassociated with the encoding of a target block to be filtered by usingthe same method as that the intra prediction part according toabove-mentioned Embodiment 1 uses, and carries out a filtering processon the intra prediction image P_(i) ^(n) by using this filter. Afterdetermining intra prediction parameters for each of all coding blocksB^(n) in the picture, for each area in which an identical filter is usedwithin the picture (each area having the same filter index), the intraprediction part 4 designs a Wiener filter which minimizes the sum ofsquared errors between the inputted image in the area and the intraprediction image (mean squared error in the target area).

The filter coefficients w of the Wiener filter can be determined from anautocorrelation matrix R_(s′s′) of an intermediate prediction imagesignal s′, and a cross correlation matrix R_(ss′) of the inputted imagesignal s and the intermediate prediction image signal s′ according tothe following equation (4). The size of the matrices R_(s′s′) andR_(ss′) corresponds to the number of filter taps determined.

w=R _(s′s′) ⁻¹ ·R _(ss′)  (4)

After designing the Wiener filter, the intra prediction part 4 expressesthe sum of squared errors in the target area for filter design in thecase of carrying out a filtering process using the Wiener filter as D1,the code amount at the time of encoding information (e.g., filtercoefficients) associated with the Wiener filter as R1, and the sum ofsquared errors in the target area for filter design in the case ofcarrying out a filtering process using a filter which is selected byusing the same method as that shown in above-mentioned Embodiment 1 asD2, and then checks to see whether or not the following equation (5) isestablished.

D1+λ·R1<D2  (5)

where λ is a constant.

When the equation (5) is established, the intra prediction part 4carries out a filtering process by using the Wiener filter instead of afilter which is selected by using the same method as that shown inabove-mentioned Embodiment 1. In contrast, when the equation (5) is notestablished, the intra prediction part carries out a filtering processby using a filter which the intra prediction part selects by using thesame method as that shown in above-mentioned Embodiment 1. Although theintra prediction part carries out the evaluation by using the sums ofsquared errors D1 and D2, this embodiment is not limited to thisexample. The intra prediction part can alternatively carry out theevaluation by using measures showing other prediction distortion values,such as the sums of the absolute values of errors, instead of the sumsof squared errors D1 and D2.

When carrying out a filtering process by using the Wiener filter, theintra prediction part 4 requires filter update information showing thefilter coefficients of the Wiener filter and indexes each indicating acorresponding filter which is replaced by the Wiener filter. Morespecifically, when the number of filters selectable in the filteringprocess using filter selection parameters is expressed as L, and indexesranging from zero to L-1 are assigned to the filters, respectively, whenthe designed Wiener filter is used for each index, a value of “1” needsto be encoded for the index as the filter update information, whereaswhen a prepared filter is used for each index, a value of “0” needs tobe encoded for the index as the filter update information. A variablelength encoding part 13 variable-length-encodes the filter updateinformation outputted thereto from the intra prediction part 4, andmultiplexes encoded data of the filter update information into abitstream.

Although the example of designing a Wiener filter which minimizes themean squared error between the inputted image and a prediction image ineach area for which an identical filter is used within a picture for thearea is shown in this embodiment, a Wiener filter which minimizes themean squared error between the inputted image and a prediction image ineach area for which an identical filter is used can be designed for eachof other specific areas each of which is not a picture. For example, theabove-mentioned design of a Wiener filter can be carried out only for acertain specific picture or only when a specific condition is satisfied(e.g., only for a picture to which a scene change detection function isadded and in which a scene change is detected).

A variable length decoding part 51 of a moving image decoding devicevariable-length-decodes the encoded data multiplexed into the bitstreamto obtain the filter update information. An intra prediction part 53carries out an intra-frame prediction process on each partition P_(i)^(n) of each coding block B^(n) to generate a intra prediction imageP_(i) ^(n), like that according to above-mentioned Embodiment 1. Whenreceiving the filter update information from the variable lengthdecoding part 51, the intra prediction part 53 refers to the filterupdate information to check to see whether or not there is an update tothe filter indicated by the corresponding index.

When determining from the result of the check that the filter for acertain area is replaced by a Wiener filter, the intra prediction part53 reads the filter coefficients of the Wiener filter which are includedin the filter update information to specify the Wiener filter, andcarries out a filtering process on the intra prediction image P_(i) ^(n)by using the Wiener filter. In contrast, for an area in which no filteris replaced by a Wiener filter, the intra prediction part selects afilter by using the same method as that which the intra prediction partaccording to above-mentioned Embodiment 1 uses, and carries out afiltering process on the intra prediction image P_(i) ^(n) by using thefilter.

As can be seen from the above description, because the moving imageencoding device in accordance with this Embodiment 2 is constructed insuch a way as to design a Wiener filter which minimizes the sum ofsquared errors between a coding block and a prediction image, and, whenthe use of this Wiener filter increases the degree of reduction inprediction errors as compared with the use of a filter which is selectedfrom one or more filters which are prepared in advance, carry out afiltering process on the prediction image by using the Wiener filter,instead of the selected filter, there is provided an advantage of beingable to further reduce prediction errors which occur locally as comparedwith above-mentioned Embodiment 1.

While the invention has been described in its preferred embodiments, itis to be understood that an arbitrary combination of two or more of theabove-mentioned embodiments can be made, various changes can be made inan arbitrary component according to any one of the above-mentionedembodiments, and an arbitrary component according to any one of theabove-mentioned embodiments can be omitted within the scope of theinvention.

INDUSTRIAL APPLICABILITY

As mentioned above, because the moving image encoding device, the movingimage decoding device, the moving image encoding method, and the movingimage decoding method in accordance with the present invention areconfigured in such a way as to, when an intra prediction unit carriesout an intra-frame prediction process to generate a prediction image byusing an already-encoded image signal in a frame, select a filter fromone or more filters which are prepared in advance according to the stateof various parameters associated with the encoding of a target block tobe filtered, and carry out a filtering process on a prediction image byusing the filter, and output the prediction image on which the filteringprocess has been carried out to a difference image generating unit, themoving image encoding device and the moving image encoding method aresuitable for use as a moving image encoding device for and a movingimage encoding method of encoding a moving image with a high degree ofefficiency, and the moving image decoding device and the moving imagedecoding method are suitable for use as a moving image decoding devicefor and a moving image decoding method of decoding an encoded movingimage with a high degree of efficiency.

EXPLANATIONS OF REFERENCE NUMERALS

1 encoding controlling part (encoding controlling unit), 2 blockdividing part (block dividing unit), 3 selection switch (intraprediction unit and motion-compensated prediction unit), 4 intraprediction part (intra prediction unit), 5 motion-compensated predictionpart (motion-compensated prediction unit), 6 subtracting part(difference image generating unit), 7 transformation/quantization part(image compression unit), 8 inverse quantization/inverse transformationpart, 9 adding part, 10 memory for intra prediction, 11 loop filteringpart, 12 motion-compensated prediction frame memory, 13 Variable lengthencoding unit (variable length encoding unit), 31 variable lengthdecoding part (variable length decoding unit), 52 selection switch(intra prediction unit and motion-compensated prediction unit), 53 intraprediction part (intra prediction unit), 54 motion-compensatedprediction part (motion-compensated prediction unit), 55 inversequantization/inverse transformation part (difference image generatingunit), 56 adding part (decoded image generating unit), 57 memory forintra prediction, 58 loop filtering part, 59 motion-compensatedprediction frame memory.

1. An image decoding device comprising: an intra prediction unit forcarrying out an intra-frame prediction process on a coding block whichis a unit for prediction process to generate a prediction image, when acoding mode for said coding block is an intra coding mode, wherein saidintra prediction unit carries out a process of generating anintermediate predicted value from reference samples according to anintra prediction parameter indicating a type of intra prediction andobtaining a final predicted value by filtering the intermediatepredicted value according to a block size of said block.
 2. An imagedecoding method comprising: an intra prediction processing step ofcarrying out an intra-frame prediction process on a coding block whichis a unit for prediction process to generate a prediction image, when acoding mode for said coding block is an intra coding mode, wherein saidintra prediction processing step includes a process of generating anintermediate predicted value from reference samples according to anintra prediction parameter indicating a type of intra prediction andobtaining a final predicted value by filtering the intermediatepredicted value according to a block size of said block.
 3. An imageencoding device comprising: an intra prediction unit for carrying out anintra-frame prediction process on a coding block which is a unit forprediction process to generate a prediction image, when a coding modefor said coding block is an intra coding mode, wherein said intraprediction unit carries out a process of generating an intermediatepredicted value from reference samples according to an intra predictionparameter indicating a type of intra prediction and obtaining a finalpredicted value by filtering the intermediate predicted value accordingto a block size of said block.
 4. An image encoding method comprising:an intra prediction processing step of carrying out an intra-frameprediction process on a coding block which is a unit for predictionprocess to generate a prediction image, when a coding mode for saidcoding block is an intra coding mode, wherein said intra predictionprocess step includes a process of generating an intermediate predictedvalue from reference samples according to an intra prediction parameterindicating a type of intra prediction and obtaining a final predictedvalue by filtering the intermediate predicted value according to a blocksize of said block.